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rhis  paper  presents  a  systematic  merit  function  approach  for  the 
comprehensive  evaluation  of  competing  military  systems.  In  this 
paper,  the  merit  function  is  defined  to  be  the  ratio  of  quantified 
system  benefit  to  system  life  cycle  cost.  System  benefit  is  measured  by  a 
unique  utility  function  that  quantifies  the  degree  to  which  a  given  system 
configuration  satisfies  an  identified  set  of  customer  requirements.  This  mea¬ 
sure  is  derived  from  the  information  contained  in  Quality  Function  Deploy¬ 
ment  tables.  The  second  portion  of  the  merit  function  is  a  life  cycle  cost 
measure,  which  can  be  developed  using  any  valid  estimation  technique.  With 
this  merit  function  approach,  the  cost  effectiveness  of  complex  systems  can 
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be  quantified.  Comparison  of  the  quantified  merit  of  competing  systems 
then  provides  for  objective  and  reliable  decision  making.  This  merit  func¬ 
tion  approach  is  demonstrated  by  an  evaluation  of  two  attack  helicopter 
configurations. 

INTRODUCTION 

According  to  accepted  finance  rules  of  commercial  business,  all  invest¬ 
ment  decisions  should  be  based  upon  some  comparison  of  discounted 
cash  inflows  and  discounted  cash  outflows  (Ross,Westerfield,  &  Jaffe). 
One  approach  is  to  compute  Net  Present  Value  (NPV),  which  is  dis¬ 
counted  cash  inflow  minus  discounted  cash  outflow.  If  NPV  >  0,  then 
the  investment  will  generate  a  cash  inflow  which  exceeds  the  cash  out¬ 
flow.  A  second  approach  to  investment  decision  making  uses  Profitabil¬ 
ity  Indices  (PI).  A  PI  is  the  ratio  of  discounted  cash  inflows  to  dis¬ 
counted  cash  outflows.  For  independent  projects,  the  decision  rule  is  to 
accept  the  project  if  PI  >  1  and  reject  if  PI  <  1,  In  effect,  a  PI  is  a  merit 
function  comparing  the  project  benefits — discounted  cash  inflows — to 
its  costs — discounted  cash  outflows. 

These  traditional  finance  rules  are  applicable  to  a  wide  variety  of 
investment  decisions.  However,  these  simple  rules  break  down  when 
applied  by  the  Department  of  Defense  (DoD)  when  considering  pro¬ 
curement  of  military  systems.  The  primary  difficulty  is  that  a  military 
system  seldom  generates  cash  inflow.  Instead,  a  military  system  gener¬ 
ally  represents  a  pure  expense  over  its  entire  life  cycle.  Using  traditional 
finance  rules,  NPV  will  always  be  less  that  zero  and  PI  will  always  be  less 
than  one.  Thus,  using  commercial  finance  rules,  a  military  project  will 
always  be  rejected  as  a  poor  investment!  Obviously  these  are  not  ac¬ 
ceptable  decision  rules  for  the  DoD. 

Several  alternative  rules  have  been  proposed  to  evaluate  military  sys¬ 
tems.  The  alternative  rules  focus  on  minimizing  either  acquisition  costs 
or,  more  appropriately.  Life  Cycle  Cost  (LCC).  In  reality,  however,  De¬ 
sign  To  Cost  (DSMC,  1986)  rules  are  not  universally  applicable.  Cost 
comparisons  are  only  appropriate  for  systems  with  similar  objectives  and 
of  equal  complexity.  Although  these  types  of  rules  are  effective  tools  for 
controlling  system  acquisitions  and  operations,  decision  rules  based  on 
cost  alone  are  inadequate  for  evaluating  competing  alternatives.  These 
rules  generally  ignore  the  benefit  inherent  in  each  military  system. 

The  benefits  of  a  military  system  are  real  although  they  typically  cannot 
be  quantified  in  dollars.  A  reasonable  benefit  measure  must  be  developed 
in  order  to  perform  a  reliable  cost-benefit  study.  The  military  invest¬ 
ment  decision  can  then  be  based  upon  an  objective  merit  function  which 
compares  the  non-monetary  system  benefit  to  its  monetary  cost. 
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The  most  difficult  part  of  the  proposed  merit  function  approach  is  to 
construct  an  appropriate  benefit  function.  One  common  approach  is  to 
define  a  set  of  technical  measures  (i.e.,  performance  parameters)  that 
can  be  measured  or  estimated  for  each  system.  A  merit  function  is  then 
defined  as  the  total  score  for  each  system;  the  larger  the  score  the 
better.  However,  incomplete  or  incorrect  formulations  of  such  merit 
functions  have  been  employed  in  the  past.  For  instance,  failure  to  nor¬ 
malize  numerical  scores  between  differing  technical  measures  often  leads 
to  performance  parameters  of  relatively  large  magnitude  that  overpower 
the  contributions  of  performance  parameters  which  are  relatively  small, 
thereby  unduly  biasing  the  overall  score.  Cost  and  risk  are  seldom  di¬ 
rectly  incorporated  into  the  function  definition.  Moreover,  customer 
requirements,  which  are  often  difficult  to  associate  with  engineering 
parameters,  are  commonly  ignored.  In  addition,  the  function  is  most 
often  linear,  which  does  not  allow  for  diminishing  marginal  returns  on 
the  merit  measure  (Harse,  1985;  Schrage,  Costello,  &  Mitlider,  (1989). 

This  paper  introduces  a  newly  developed  function  to  quantify  the 
benefits  of  a  complex  engineered  system.  This  function  overcomes  pre¬ 
vious  shortcomings  and  it  incorporates  direct  consideration  of  customer 
requirements.  Conceptual  development  of  this  measure  borrows  heavily 
from  the  matrix  techniques  of  Quality  Function  Deployment  (Sullivan, 
1986).  Quality  Function  Development  (QFD)  methods,  developed  in 
Japan  in  the  1970s,  are  rooted  in  a  product  development  philosophy 
emphasizing  customer-driven  design.  This  method  employs  graphical 
quality  engineering  tools  that  map  the  “voice  of  the  customer”  into 
product  and  process  design  characteristics.  A  QFD  method  is  then  used 
to  ensure  the  key  product  development  objectives  of  quality,  cost,  and 
timeliness  are  retained  throughout  product  development  and  manufac¬ 
turing. 

The  second  major  element  of  the  proposed  merit  function  is  a  Life 
Cycle  Cost  (LCC)  measure.  Life  Cycle  Cost  is  simply  the  summation  of 
all  expenditures  required  from  conception  of  a  system  until  it  is  phased 
out  of  operational  use.  Historically,  a  low  initial  acquisition  cost  has  not 
assured  a  low  LCC.  In  fact,  the  opposite  is  true.  This  trend  is  explained 
by  the  fact  the  majority  of  LCC  (at  least  for  military  systems)  is  usually 
in  operations  and  support  (Q&S).  The  greatest  potential  opportunity 
for  cost  reduction  in  the  Department  of  Defense  is  now  recognized  as 
control  of  the  cost  of  system  support.  This  cost  element  will  be  invisible 
in  the  selection  process  and  so  cannot  be  controlled  unless  a  LCC  model 
is  used. 

With  system  benefit  and  cost  quantified,  an  overall  merit  function  is 
defined  as  the  ratio  of  system  benefit  to  its  LCC.  Two  merit  functions 
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are  developed.  The  first  function  assumes  that  no  existing  system  is 
available.  The  second  function  assumes  the  systems  under  consideration 
will  replace  an  existing  system.  In  this  case,  the  merit  function  quantifies 
the  incremental  change  in  benefit  and  cost  compared  with  the  baseline 
system.  For  the  merit  functions  employed  in  this  paper,  the  decision  rule 
is  simply  to  maximize  system  merit. 

A  brief  outline  of  this  paper  follows.  The  merit  function  is  developed 
and  the  appropriate  decision  rule  is  presented.  Next,  the  new  benefit 
function  is  developed.  Construction  of  the  QFD  Tables  and  calculation 
of  the  benefit  function  are  detailed.  A  brief  discussion  of  the  appropri¬ 
ate  cost  functions  follows.  Finally,  application  of  the  merit  function  ap¬ 
proach  is  demonstrated  by  an  actual  application  to  the  evaluation  of  two 
attack  helicopters. 

THE  MERIT  FUNCTION 

The  merit  function  is  defined  as  a  single  number  which,  when  properly 
determined,  reflects  the  ratio  of  benefits  derived  to  dollars  spent.  For 
the  proposed  merit  function,  the  system  with  greatest  merit  is  deemed 
the  most  desirable.  Let  M  represent  overall  merit,  B  derived  benefit, 
and  C  Life  Cycle  Cost.  The  absolute  merit  of  any  given  configuration 
then  is; 


M  =  B/C  (1) 

This  function  provides  the  means  for  objective  comparison  of  two  or 
more  complex  configurations  when  no  baseline  system  exists.  A  large 
system  merit  is  preferable  to  a  small  one. 

This  relationship  between  benefit  and  cost  is  graphically  represented 
in  Figure  1.  In  this  graph,  cost  is  plotted  on  the  horizontal  axis  and 
benefit  on  the  vertical.  Note  that  merit,  Eq.  (1),  represents  the  slope  of 
the  line  connecting  the  plotted  merit  value  and  the  graph’s  origin.  Let 
the  point  labeled  1  represent  the  merit  of  a  baseline  system.  If  a  con¬ 
figuration  were  introduced  for  which  both  benefit  and  cost  are  increased 
proportionately,  overall  merit  remains  the  same.  Such  a  point  is  labeled 
2  in  Figure  1,  and  will  always  lie  on  the  same  line  connecting  point  1  and 
the  origin.  If  benefit  were  increased,  but  cost  remained  the  same,  the 
new  system’s  merit  would  be  larger  (the  new  point  will  fall  above  the 
shaded  region  in  Figure  1).  Conversely,  if  benefit  remained  constant 
while  cost  increased,  the  new  system’s  merit  would  be  less  (the  new 
point  will  lie  in  the  shaded  region  of  Figure  1).  The  defined  merit  func¬ 
tion  becomes  particularly  useful  when  evaluating  realistic  problems  where 
both  benefit  and  cost  are  sensitive  to  system  design.  When  such  is  the 
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Figure  1.  Graphical  Representation  of  the  Merit  Function. 


case,  it  is  difficult  to  predict,  a  priori,  the  change  in  merit.  The  decision 
rule  remains  constant,  however,  and  the  system  with  largest  merit  is 
judged  most  desirable.  Returning  to  Figure  1,  points  1  and  2  have  the 
same  merit;  point  4’s  merit  is  less  than  that  of  1  and  2;  and  point  3  has 
the  highest  merit  of  all.  Though  the  above  circumstances  are  transpar¬ 
ent,  the  problem  remains  to  define  a  systematic  procedure  for  quantify¬ 
ing  both  benefit  and  cost. 

The  merit  function  has  direct  economic  interpretation.  It  is  a  mea¬ 
sure  of  system  benefit  per  dollar  expended.  A  rational  decision  maker 
chooses  to  maximize  the  benefit  obtained  for  each  dollar  spent,  and 
selects  the  system  with  highest  merit.  Consider  the  inverse  of  the  merit 
function,  P. 


P  =  1/M  =  C/B  (2) 

This  function  measures  dollars  spent  to  achieve  a  particular  level  of 
system  benefit.  The  variable  P  represents  dollar  cost  per  unit  of  system 
merit.  In  other  words,  P  is  simply  the  price  of  a  unit  of  system  benefit. 
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In  this  case,  a  rational  decision  maker  selects  the  system  with  smallest 
price  per  increment  of  system  merit,  which  again  is  the  system  for  which 
merit  is  greatest. 

QUANTIFYING  BENEFIT 

In  this  section,  a  method  for  quantifying  system  benefit  is  developed.  A 
Benefit  Function  is  defined  to  measure  the  degree  to  which  a  given 
system  configuration  satisfies  customer  requirements.  To  organize  the 
data,  the  methodology  uses  a  QFD  Table.  The  Benefit  Function  is  de¬ 
rived  in  part  from  the  QFD  Table,  which  relates  the  engineering  perfor¬ 
mance  of  a  system  to  specific  customer  requirements. 

In  essence,  a  single  measure  of  benefit  is  determined  (i.e.,  a  single 
number)  for  each  candidate  product  design.  This  measure,  or  score, 
represents  the  degree  to  which  each  candidate  balances  conflicting  de¬ 
sign  (i.e.,  customer)  requirements.  This  measure  is  expressed  as  a  per¬ 
centage  difference  from  an  ideal  system  (i.e.,  a  system  that  achieves 
specified  target  values  for  all  of  the  customer’s  stated  requirements)  and 
evaluation  is  biased  by  the  customer’s  stated  priorities. 

CONSTRUCTION  OF  THE  PLANNING  TABLE 

The  first  step  in  quantifying  system  benefit  is  the  construction  of  the 
Planning  Table.  The  general  form  of  the  planning  table  is  depicted  in 
Figure  2.  Four  lists  are  compiled  in  order  to  begin  construction  of  this 
table.  The  first  is  a  list  of  the  customer’s  requirements,  that  is,  a  list  of 
the  desired  characteristics  of  the  final  system  stated  in  the  customer’s 
own  words.  The  second  list  is  a  set  of  “importance  weighting  factors” 
used  to  prioritize  each  of  the  requirements.  These  factors  must  also  be 
solicited  from  the  customer.  The  third  is  a  candidate  set  of  perfor¬ 
mance/analysis  parameters  to  be  measured  or  predicted  and  compared 
with  their  corresponding  target  levels.  These  target  levels  are  chosen  to 
represent  the  ideal  system  and  would  usually  reflect  state-of-the-art  tech¬ 
nology.  The  performance/analysis  parameters  are  to  be  used  to  evaluate 
candidate  system  designs  in  relation  to  the  stated  customer  require¬ 
ments.  The  fourth  list  needed  to  construct  the  planning  table  is  a  list  of 
competitive  systems  and/or  design  options. 

Once  these  four  lists  are  compiled,  the  first  iteration  of  the  Planning 
Table  can  be  constructed.  There  are  four  primary  components  to  the 
table.  First  is  the  Relationship  Matrix,  labeled  Table  A  in  Figure  2.  This 
table  is  used  to  assess  and  document  interactions  between  the  customer’s 
stated  requirements  and  the  selected  performance/analysis  parameters. 
Interactions  (i.e.,  each  entry  in  the  matrix)  are  typically  classified  as 
either  strong,  moderate,  or  weak.  The  Relationship  Matrix,  once  com- 
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Figure  2.  QFD  Planning  Table  Used  in  Quantifying  Benefits. 


plete,  is  used  to  determine  whether  or  not  the  selected  performance 
characteristics  can  adequately  measure  satisfaction  of  the  customer  re¬ 
quirements.  If  relationships  to  a  given  customer  requirement  are  pre¬ 
dominantly  weak,  additional  performance  parameters  are  introduced  to 
ensure  compliance  with  the  requirement  can  be  evaluated.  Similarly,  the 
matrix  entries  can  be  used  to  identify  a  minimum  set  of  performance 
parameters  needed  to  be  evaluated  and  tracked. 

The  second  component  of  the  planning  table  is  the  Customer  Rating 
Column,  labeled  Table  B  in  Figure  2.  A  list  of  “importance  weighting 
factors”  associated  with  the  requirements  list  is  solicited  from  the  cus¬ 
tomer  and  used  to  fill  out  this  column.  These  weighting  factors  are  to  be 
ordered  so  the  largest  numbers  represent  the  most  important  require¬ 
ments.  The  factors  are  later  normalized  so  that  their  sum  equals  1.0. 
This  constraint  ensures  consistency  between  differing  sets  of  weighting 
factors. 

The  next  component  of  the  Planning  Table  is  the  Performance  Com¬ 
parison  Matrix,  Table  C  in  Figure  2.  In  this  matrix,  results  of  analytic  or 
numerical  predictions  or  experimentation  are  tabulated  for  the  system 
under  evaluation  and  its  competitors.  Target  levels  for  each  of  the  per¬ 
formance/analysis  parameters  are  also  tabulated  in  this  matrix.  With 
performance  parameters  tabulated,  the  systems  are  evaluated  as  to  their 
satisfaction  of  the  customer  requirements.  The  results  of  this  evaluation 
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are  recorded  in  the  Requirement  Evaluation  Matrix,  Table  D  of  Figure  2. 
The  Planning  Table  is  then  complete. 

DERIVATION  OF  THE  BENEFIT  FUNCTION 

The  Benefit  Function  is  an  objective,  numerical  measure  derived  from 
the  Planning  Table  data.  This  function  evaluates  system  performance 
relative  to  the  complex,  and  possibly  conflicting,  customer  requirements. 
There  are  three  primary  components  to  the  Benefit  Function.  The  first 
component  is  the  utility  measure,  which  is  used  to  compare  system  per¬ 
formance  to  target  values  in  a  non-dimensional  format.  The  second  com¬ 
ponent  is  the  customer  satisfaction  calculation,  which  determines  how 
well  a  system  satisfies  individual  customer  requirements.  Finally,  the 
benefit  calculation  combines  the  customer  satisfaction  results  with  the 
“importance  weighting  factors”  to  develop  an  overall  score  for  the  sys¬ 
tem.  Thus,  the  Benefit  Function  measures  the  degree  to  which  a  system 
satisfies  the  weighted  customer  requirements. 

The  first  component  of  the  Benefit  Function  develops  a  non-dimen¬ 
sional  utility  measure  from  the  data  contained  in  the  Performance  Com¬ 
parison  Matrix.  It  is  based  upon  the  ratio  of  each  performance  param¬ 
eter  to  the  corresponding  target  level.  Let  the  numerical  entries  in  the 
performance  comparison  matrix  be  represented  by  the  following  matrix. 

D  =  [djJ  i  =  1,...,  n-Hl  j  =  1,  ....,  p  (3) 

where  n-t-1  represents  target  data  plus  the  number  of  proposed  systems 
and  its  competitors,  and  p  is  the  number  of  performance  parameters 
measured.  Note  that  the  target  levels  for  the  performance  parameters 
are  given  in  the  first  row  of  the  data  matrix.  In  other  words,  parameter 
targets  are  represented  by  dj.,  j  =  1,  ...,  p. 

The  utility  measure  is  applied  to  the  systems  represented  by  the  rows 
in  the  data  matrix,  D.  The  resulting  utility  matrix,  U,  is  of  the  same 
dimension  as  the  data  matrix  D  in  Eq.  (3). 


u  =  [“.il 

where 


(4) 


u .  = 


■J 


if  j-th  target  level  is  a  desired  lower  limit 
if  j-th  target  level  is  a  desired  upper  limit 


(5) 


If  Uj.  <  1,  then  the  performance  parameter  does  not  meet  the  target 
level,  and  similarly,  if  Uj.  >  1,  the  performance  exceeds  the  target  level. 
If  u,.  =  1,  then  the  j-th  parameter  is  equal  to  its  target  level.  The  first 
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row  of  the  utility  matrix  U,  is  Uj.  =  V  =  1,  j  =  1,  p  which 
represents  the  ideal  utility  of  each  performance  parameter.  Note  if  the  j- 
th  target  level  is  a  desired  lower  limit,  then  the  j-th  parameter  must 
exceed  the  target  for  u..  >  1.  Likewise,  if  the  j-th  target  level  is  a  desired 
upper  limit,  then  the  j-th  parameter  must  be  less  than  the  target  for  Uj. 
>  1. 

In  Eq.  (5),  the  square  root  function  is  used  to  provide  decreasing 
marginal  returns  to  the  utility  measure.  The  incremental  utility  gained 
decreases  as  a  performance  parameter  approaches  and  surpasses  its  tar¬ 
get  level.  It  is  important  that  diminishing  marginal  return  behavior  is 
ensured  so  the  Benefit  Function  is  consistent  with  traditional  economic 
theory.  Previous  studies  have  recognized  the  need  for  diminishing  mar¬ 
ginal  return  behavior,  but  these  studies  were  unable  to  achieve  this 
property  (Harse,  Schrage,  et  al.).  Using  diminishing  marginal  returns, 
the  benefit  of  a  system  with  all  the  performance  parameters  at  or  near 
the  target  levels  is  greater  than  the  benefit  of  a  system  where  several 
parameters  greatly  exceed  their  targets  while  others  fall  significantly 
short  of  the  desired  level. 

After  the  utility  matrix  is  computed  by  Eq.  (4)  and  Eq.  (5),  the  cus¬ 
tomer  satisfaction  calculation  is  performed.  This  calculation  determines 
how  well  the  system  satisfies  each  of  the  customer  requirements  listed  in 
the  Relationship  Matrix,  Table  A  in  Figure  2.  Specifically,  the  satisfaction 
for  each  requirement  is  a  summation  of  the  system  utility  components 
weighted  by  the  interactions  between  each  parameter  and  the  specific 
customer  requirement. 

To  perform  the  customer  satisfaction  calculation,  the  symbolic  Rela¬ 
tionship  Matrix  must  be  translated  into  a  numerical  matrix.  For  each 
strong  interaction,  a  value  of  3  is  assigned  to  the  matrix  element;  2  to 
each  moderate  interaction;  1  is  assigned  to  each  weak  interaction;  fi¬ 
nally,  0  is  used  to  indicate  no  interaction.  The  resulting  numerical  rela¬ 
tionship  matrix  is  given  by 


X  =  [\j]  k  =  1,  ...,  r  j  =  1,  ...,  p  (6) 


where  p  is  defined  earlier,  r  is  the  number  of  customer  requirements 
and 


x 


kj 


3  for  a  strong  interaction 
2  for  a  moderate  interaction 
1  for  a  weak  interaction 
0  for  no  interaction 


(7) 


With  X  defined  in  Eq.  (6)  and  Eq.  (7),  the  customer  satisfaction 
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calculation  can  be  performed.  The  raw  customer  score  is  computed  as 

S  =  XUT  (8) 

where  S  is  dimensioned  r  x  n+1  and  the  superscript  T  denotes  the 
matrix  transpose.  Customer  satisfaction  is  computed  by 

S  =  100%  X  S,/S,j  i  =  l,...,  n+1 

(9) 

where  the  first  column  of  S,  S^,,  k  =  1,  ...,  r,  represents  the  satisfaction 
of  a  system  which  identically  satisfies  all  parameter  target  levels.  Thus, 
is  the  customer  satisfaction  of  the  target,  or  “ideal,”  system  for  each 
customer  requirement.  The  customer  satisfaction  results,  Eq.  (9),  are 
used  to  construct  the  Requirement  Evaluation  Matrix,  Table  D  in  Figure 
2.  For  a  given  requirement  k,  if  S^.>  100%,  then  the  i-th  system  exceeds 
the  satisfaction  level  of  the  ideal  system.  Similarly,  if  S^j<  100%,  then 
the  i-th  system  falls  short  of  the  satisfaction  of  the  target  system. 

The  final  step  of  the  analysis  is  to  compute  system  benefit.  The  ben¬ 
efit  calculation  is  a  summation  of  customer  satisfaction  levels,  Eq.  (8), 
weighted  by  the  Customer  Requirement  Ratings,  Table  B  in  Figure  2. 
This  calculation  is  expressed  as 


B  =  R'^S  (10) 

where  R  is  the  vector  of  “importance  weighting  factors”  for  the  cus¬ 
tomer  requirements.  As  defined  in  Eq.  (10),  the  benefit  vector  is  dimen¬ 
sioned  1  x  n+1,  where  n  is  defined  earlier.  The  first  element  of  B,  Bj,  is 
the  benefit  of  the  ideal  system.  The  remaining  elements  of  B,  B^,  ..., 
B^^j,  are  the  benefit  values  of  the  candidate  configurations. 

The  benefit  calculation  results  can  be  conveniently  expressed  as  a 
percentage  of  the  ideal  system  benefit. 

B.  =  100%  X  B/B^  i  =  1,...,  n  +  1  (11) 

Note  that  Bj  =  100%.  Any  system  with  Bj  >  100%  exceeds  the  benefit 
of  an  ideal  system,  where  the  ideal  sj^stem  matches  all  target  levels  of 
performance.  On  the  other  hand,  if  Bj  <  100%,  the  candidate  system 
does  not  meet  the  ideal  benefit  level.  target  levels  are  selected  to 
represent  a  state-of-the-art  system,  then  B.  <  100%  will  be  the  typical 
result. 
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QUANTIFYING  COST 

Life  Cycle  Cost  (LCC)  can  be  determined  using  a  wide  variety  of  tech¬ 
niques.  Three  common  approaches  are  parametric  analysis,  determina¬ 
tion  by  analogy,  and  the  so  called  “bottom-up”  technique.  Choice  is 
determined  largely  by  the  type  of  system  being  studied  and  the  available 
database  (DSMC,  1986). 

Parametric  analysis  of  LCC  relies  heavily  on  statistical  cost  estimating 
relationships.  These  relationships  must  either  already  be  available  or  be 
developed  from  available  data.  In  most  cases,  data  representing  a  broad 
array  of  related  systems  must  be  obtained.  Regressions  are  then  con¬ 
structed  which  relate  system  LCC  to  one  or  more  characteristic  param¬ 
eters,  such  as  vehicle  and  subsystem  weights.  This  approach  is  most 
useful  for  conceptual  design.  However,  it  can  be  difficult  to  obtain  an 
appropriate  and  up-to-date  database  with  statistical  relevance,  and  it  is 
sometimes  difficult  to  determine  the  statistical  significance  of  publicly 
available  cost  estimating  relationships. 

Life  cycle  cost  estimation  by  analogy  is  primarily  used  to  calibrate  the 
results  of  parametric  analysis.  With  this  approach,  the  LCC  of  a  system 
is  determined  by  analogy  to  available  cost  data  on  an  existing  and  simi¬ 
lar  system.  Adjustments  are  made  to  various  cost  elements  to  account 
for  differences  between  the  old  and  new  systems.  Analogy  approaches 
are  especially  useful  when  an  insufficient  database  exists  to  develop 
statistically  significant  cost  estimating  relationships. 

The  third  LCC  estimation  technique  to  be  discussed  is  sometimes 
referred  to  as  a  “bottom-up,”  or  engineering  data,  approach.  In  general, 
the  cost  contributions  of  each  subsystem  component  are  estimated  from 
knowledge  of  the  component’s  design,  material  properties,  and  intended 
use.  The  primary  drawback  of  this  approach  is  that  it  can  be  labor- 
intensive  and  time-consuming. 

Since  there  is  a  wealth  of  information  available  in  the  literature  for 
LCC  estimation,  this  subject  will  not  be  dealt  with  in  any  greater  detail. 

EVALUATION  OF  TWO  ATTACK  HELICOPTERS 
Opportunities  for  improved  AH-IW  Attack  Helicopter  mission  effec¬ 
tiveness,  new  mission  capability,  and  improved  survivability  are  currently 
being  exploited  by  the  United  States  Marine  Corps  (USMC).  In  particu¬ 
lar,  procurement  of  a  night  targeting  system  is  underway,  along  with  a 
navigation  system  upgrade  and  improvements  in  the  electronic  warfare 
suite.  Evolution  of  the  USMC  AH-1  is  expected  to  continue  well  into 
the  next  century.  However,  the  many  benefits  of  incorporating  advanced 
technology'  will  be  offset  by  increased  vehicle  weight  and  a  correspond¬ 
ing  reduction  in  payload  capability  of  the  vehicle. 
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Table  1. 


USMC  REQUIREMENTS  LIST  WITH  COMPOSITE  WEIGHTINGS 


MARINE  CORPS  REQUIREMENTS 

RELATIVE 
WEIGHTING 
(1.0  Highest) 

Improved  Reliability  and  Maintainability 

1.00 

Increased  Speed 

0.94 

Increased  Maneuverability  and  Agility 

0.90 

Harder  to  Kill 

0.87 

Reduced  Vibrations  and  Loads 

0.86 

Carry  More 

0.79 

Reduced  Operations  and  Support  Costs 

0.70 

Harder  to  Detect 

0.50 

Operate  Over  the  Horizon 

0.49 

Easier  to  Fly 

0.49 

Increased  Endurance 

0.39 

The  AH-IW  manufacturer,  Bell  Helicopter  Textron,  Inc.  (BHTI),  in 
recognition  of  the  trend  toward  decreasing  payload  capability  and  degraded 
performance,  initiated  an  independent  research  and  development  (IR&D) 
program  to  first  evaluate,  and  then  demonstrate,  the  feasibility  of  applying 
available  BHTI  four-bladed,  bearingless  main  rotor  technology,  in  combina¬ 
tion  with  an  uprated  drive  system,  to  achieve  significant  improvements  in 
AH-IW  vehicle  performance  and  payload  capability.  The  technology  dem¬ 
onstrator  developed  by  BHTI  is  referred  to  as  the  4BW.  The  evaluation  of 
the  cost  effectiveness  of  adopting  the  4BW  in  place  of  the  AH-IW  is  an 
example  of  a  complex  military  management  decision.  This  problem  pro¬ 
vides  an  excellent  opportunity  for  the  application  of  the  developed  cost- 
benefit  methodology.  For  simplicity,  risk  is  not  included  in  the  presentation 
of  this  example.  A  detailed  cost-benefit  analysis  of  the  two  systems  was 
documented  by  Corban  et  al.  in  1991. 

A  survey  of  knowledgeable  military  and  civilian  personnel  was  conducted 
and  resulted  in  the  USMC  requirements  listed  in  Table  1.  The  relative 
weightings  were  obtained  by  averaging  the  weights  provided  by  the  sur¬ 
veyed  personnel.  The  performance  measures  listed  in  Table  2  were  used 
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Table  2 

SELECTED  PERFORMANCE  MEASURES  FOR  BENEFIT  ASSESSMENT 

Performance  Measure 

Target 

Value 

(1) 

Isolateid  Rotor  Figure  of  Merit  at  Operating  Blade  Loading 
(hover,  Navy  hot  day  conditions,  battle  station  weights, 
escort/anti-arnnor  ordnance  load) 

0.85 

(2) 

Hover  Ceiling  (battle  station  weights,  escort/anti-armor 

ordnance  load,  2082  hp  transmission  limit  (AH-1W),  2625  hp  (4BW)) 

15,000  ft 

(3) 

Maximum  Rate  of  Climb  (sea  level  standard,  battle 
station  weights,  escort/anti-armor  ordnance  load) 

4,000  ft/min 

(4) 

Service  Ceiling  (standard  day,  battle  station  weights, 
escort/anti-armor  ordnance  load) 

25,000  ft 

(5) 

Maximum  Vertical  Rate  of  Climb  (VROC) 

(Navy  hot  day  conditions,  battle  station  weights, 
escort/anti-armor  ordnance  load) 

1 ,5000  ft/min 

(6) 

Dash  Speed  (Navy  hot  day  conditions,  battle  station 

weights,  escort/anti-armor  ordnance  load,  intermediate  rated  power) 

180  knots 

(7) 

Cruise  Speed  (Navy  hot  day  conditions,  battle  station  weights 
escort/anti -armor  ordnance  load,  max  continuous  power) 

160  knots 

(8) 

Radius  Ordnance  Factor 

4,000  Ib-nm  per  100 

(9) 

Station  Ordnance  Factor 

5,000  Ib-hrs 

(10) 

Maneuverability/Agility  Assessment 

(Note  overall  score  is  the  sum  of  tabulated  scores  at 

take-off  and  battle  station  weights 

1.0 

(11) 

Structures  (determined  as  the  percentage  difference 
from  limits  of  the  allowable  maneuvering  loads  envelope) 

1.0 

(12) 

Vibrations/Dynamic  Loads  (qualitative  assessment) 

1.0 

(13) 

Acoustic  Signature  (qualitative  score  based  on  first-order  main 
rotor  signature  estimates) 

1.0 

(14) 

Impact  on  Vulnerability  (qualitative  assessment) 

1.0 

(15) 

Handling  Qualitites  (qualitative  assessment) 

1.0 

to  quantify  the  performance  of  the  two  candidate  helicopters.  This  list 
was  influenced  by  standard  engineering  practice,  the  identified  require¬ 
ments,  and  the  engineering  tools  available  for  use  in  the  assessment. 
Target  values  were  established  by  a  combination  of  military  specifica¬ 
tions,  current  AH-IW  performance,  and  state-of-the-art  rotorcraft  tech¬ 
nology.  Those  measures  were  subjectively  evaluated  based  on  available 
data,  whereas  the  remaining  were  evaluated  quantitatively  using  com¬ 
puter  analysis  tools. 
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Figure  3.  Planning  Table  for  AH-IW  Example. 


14  -  Winter  1995 


Acquisition  Review  Quarterly 


Configuration  B-4BW  .743  |l2,250|  2,707  [24.937|  1>453  |  171  |  166  |  861  [1,076  |  .823  |  .666 


A  Novel  Cost-Benefit  Analysis 
for  Evaluation  of  Complex  Military  Systems 


Figure  4.  Utility  Matrix  and  Benefit  Score  for  AH-IW  Example. 
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The  customer  requirements  and  performance  parameters  are  as¬ 
sembled  into  a  Planning  Table  as  depicted  Figure  3.  The  performance 
parameters  of  Table  2  are  distributed  horizontally  across  the  top  of  the 
matrix.  The  prioritized  requirements  list  of  Table  1  is  distributed  verti¬ 
cally  along  the  left-hand  side  of  the  upper  matrix.  These  requirements 
are  listed  in  order  of  importance,  starting  from  the  top,  as  indicated  by 
weightings  entered  in  the  second  column.  Interactions  between  indi¬ 
vidual  requirements  and  performance  measures  1-15  are  then  subjec¬ 
tively  characterized  as  either  strong  (S),  medium  (M),  weak  (W),  or 
relatively  nonexistent  (N),  and  entered  into  the  Relationship  Matrix. 
Next,  the  candidate  helicopter  configurations  are  listed  vertically  in  the 
first  column  of  the  Performance  Comparison  Matrix,  also  of  Figure  3. 
Last,  the  numerical  results  and  target  values  assigned  to  each  of  the 
fifteen  performance  measures  are  entered  in  the  lower  matrix. 

With  the  Planning  Table  complete,  the  algorithm  defined  by  Eq.  (11) 
and  Eq.  (12)  is  used  to  construct  the  Utility  Matrix  shown  in  Figure  4. 
Next,  the  interaction  classifications  (strong,  medium,  weak,  or  nonexist¬ 
ent)  are  translated  into  numerical  values  using  Eq.  (13).  They  serve  to 
prevent  a  good  score  for  one  requirement  from  influencing  an  unrelated 
requirement’s  total  score.  For  example,  the  score  for  acoustic  signature 
should  not  improve  benefit  attributed  to  a  configuration  that  is  “easier 
to  fly”  (the  second-to-last  requirement),  since  the  two  requirements  share 
little  relationship.  The  outcome  of  the  customer  satisfaction  calculation, 
Eq.  (15)  and  Eq.  (16),  is  presented  graphically  in  Figure  5.  This  graph 
represents  the  Requirement  Evaluation  Matrix:  Table  D,  Figure  2.  Note 
that  the  4BW  helicopter  configuration  proves  superior  to  the  AH-IW  in 
satisfying  each  individual  customer  requirement. 

Finally,  the  Benefit  Measures  for  the  two  systems  are  then  ealculated 
using  Eq.  (17)  and  Eq.  (18).  Note  in  this  calculation  the  normalized 
weightings  have  been  scaled  so  that  the  sum  is  one.  This  overall  score  is 
measured  relative  to  an  ideal  system  which  achieves  all  target  values  (i.e. 
the  ideal  system  seores  100%).  These  overall  scores  are  presented  on 
the  right-hand  side  of  Figure  4.  Based  on  this  assessment,  the  4BW 
exhibits  an  18.8%  improvement  in  benefit  over  the  AH-IW.  That  is,  the 
4BW  achieves  83.6%  of  the  overall  target  level,  while  the  AH-IW  achieves 
only  64.8%. 

Preliminary  LCC  estimates  are  generated  from  a  generic  AH-1  cost 
model  that  includes  the  cost  elements  depicted  in  Figure  6.  Two  differ¬ 
ent  operating  scenarios,  readiness  and  contingency,  are  considered,  and 
estimates  formed  for  each.  Readiness  level  assumes  no  combat  over  the 
aircraft  life.  The  contingency  level  assumes  various  combat  engagements 
over  the  life  of  the  system. 
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Figure  5.  Percentage  of  Target  Values  Achieved. 


Rough-order-of-magnitude  estimates  of  LCC  for  the  AH-IW  and  4BW 
helicopters  are  presented  in  Table  3  in  1990  billions  of  dollars  for  both 
the  readiness  and  contingency  scenarios.  These  estimates  represent  a 
lower  bound  on  LCC.  Cost  estimates  for  all  categories  of  Figure  6  were 
not  available.  Acquisition  of  the  4BW  requires  approximately  200  mil¬ 
lion  dollars  of  additional  expenditure  over  the  life  of  the  system. 
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Figure  6.  Cost  Categories  for  all  AH-1  LCC  Model. 


The  calculation  of  comparative  AH-IW  and  4BW  merit  is  presented 
in  Table  3.  These  calculations  are  based  on  preliminary  assessment  of 
proposed  four-bladed  main  rotor  system  benefit  and  lower  bound  esti¬ 
mates  for  AH-IW  and  4BW  life  cycle  cost  (the  4BW  is  assumed  to 
provide  5%  reduction  in  maintenance  and  spares  costs).  Based  on  this 
preliminary  assessment,  the  4BW’s  benefit  outweighs  fleet  conversion 
cost  in  both  readiness  and  contingency  scenarios.  The  4BW’s  merit  ex¬ 
ceeds  that  of  the  AH-IW  by  17%  in  the  readiness  scenario,  and  20%  in 
contingency.  Procurement  of  the  4BW  thus  yields  a  20%  higher  level  of 
mission  effectiveness  (in  the  contingency  scenario)  per  dollar  spent.  This 
preliminary  positive  result  justifies  a  recommendation  for  further  engi¬ 
neering  development  of  the  4BW  and  more  comprehensive  evaluation 
of  its  potential  value  to  the  Marine  Corps. 

SUMMARY 

This  paper  presents  a  merit  function  approach  for  cost-benefit  analysis 
of  high  tech  systems,  which  relies  on  a  comprehensive  measure  of  sys- 
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Table  3 

CALCULATION  OF  AH-IW  AND  4BW  MERIT  BASED  ON  PRELIMINARY 
PERFORMANCE  AND  COST  ESTIMATES 


Helicopter 

Configurations 

Preliminary 

Benefit 

Score 

Life  Cycle  Cost 

Estimated  Floor  In  1990  -  $  Biltlon  1 

MERIT 

Readiness 

Level 

Contingency! 

Level 

Readiness 

Level 

Contingency 

Level 

Configuration  A  -  AH-1 W 

64.8 

1.764 

2.295 

36.74 

28.24 

Configuration  B  -  4BW 

83.6 

1.940 

2.466  ^ 

43.09 

17%  Increase 

33.90  i 

20%  Increase 

tern  utility.  The  benefit  function  is  derived  in  part  from  Quality  Func¬ 
tion  Deployment  Tables,  which  allow  for  the  measurement  of  both  mon¬ 
etary  and  non-monetary  attributes.  The  cost  component  of  the  merit 
function  is  system  life  cycle  cost.  Application  of  the  methodology  was 
demonstrated  by  an  evaluation  of  two  competing  attack  helicopter  con¬ 
figurations.  This  decision  example  demonstrates  the  practical  applica¬ 
tion  of  this  methodology  to  the  evaluation  of  a  complex  system  employ¬ 
ing  advanced  technology. 
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